Method for transferring control of an autonomous vehicle to a remote operator

ABSTRACT

One variation of a method for transferring control of an autonomous vehicle to a remote operator includes: accessing a specification for triggering manual control of autonomous vehicles; identifying a road segment, within a geographic region, exhibiting characteristics defined by the specification; and associating a location of the road segment, represented in a navigation map, with a remote operator trigger. The method also includes, at the autonomous vehicle operating within the geographic region: autonomously navigating along a route; transmitting a request for manual assistance to the remote operator in response to approaching the location associated with the remote operator trigger; transmitting sensor data to a remote operator portal associated with the remote operator; and executing a navigational command received from the remote operator via the remote operator portal; and resuming autonomous navigation along the route after passing the location.

CROSS-REFERENCE TO RELATED APPLICATIONS

This Application claims the benefit of U.S. Provisional Application No.62/592,806, filed on 30Nov. 2017, which is incorporated in its entiretyby this reference.

TECHNICAL FIELD

This invention relates generally to the field of autonomous vehicles andmore specifically to a new and useful method for transferring control ofan autonomous vehicle to a remote operator in the field of autonomousvehicles.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 is a flowchart representation of a method;

FIG. 2 is a flowchart representation of one variation of the method;

FIGS. 3A, 3B, and 3C are flowchart representations of variations of themethod; and

FIG. 4 is a flowchart representation of one variation of the method.

DESCRIPTION OF THE EMBODIMENTS

The following description of embodiments of the invention is notintended to limit the invention to these embodiments but rather toenable a person skilled in the art to make and use this invention.Variations, configurations, implementations, example implementations,and examples described herein are optional and are not exclusive to thevariations, configurations, implementations, example implementations,and examples they describe. The invention described herein can includeany and all permutations of these variations, configurations,implementations, example implementations, and examples.

1. METHOD

As shown in FIGS. 1, 2, and 3A, a method for transferring control of anautonomous vehicle to a remote operator includes, at a computer system:accessing a corpus of driving records of a fleet of autonomous vehiclesoperating within a geographic region in Block S110; identifying a roadsegment, within the geographic region, associated with a frequency oftransitions, from autonomous operation to local manual operationtriggered by local operators occupying autonomous vehicles in the fleet,that exceeds a threshold frequency based on the corpus of drivingrecords in Block S120; and associating a location of the road segment,represented in a navigation map, with a remote operator trigger in BlockS130. The method also includes, at the autonomous vehicle operatingwithin the geographic region: autonomously navigating along a route inBlock S140; transmitting a request for manual assistance to the remoteoperator in Block S150 in response to approaching the locationassociated with the remote operator trigger; transmitting sensor data toa remote operator portal associated with the remote operator in BlockS152; executing a navigational command received from the remote operatorvia the remote operator portal in Block S154; and resuming autonomousnavigation along the route after passing the location in Block S160.

One variation of the method shown in FIG. 3C further includes, at theremote computer system: accessing a corpus of historical trafficaccident data of human-operated vehicles involved in traffic accidentswithin a geographic region in Block S110; identifying a road segment,within the geographic region, associated with a frequency of trafficaccidents that exceeds a threshold frequency based on the corpus ofhistorical traffic accident data in Block S120; and associating alocation of the road segment, represented in a navigation map, with aremote operator trigger in Block S130.

Another variation of the method shown in FIGS. 3A, 3B, and 3C furtherincludes, at the remote computer system: accessing a specification fortriggering manual control of autonomous vehicles in Block S110;identifying a road segment, within a geographic region, exhibitingcharacteristics defined by the specification in Block S120; andassociating a location of the road segment, represented in a navigationmap, with a remote operator trigger in Block S130.

2. APPLICATIONS

Generally, Blocks of the method can be executed by a computer system(e.g., a computer network, a remote server) to preemptively annotate anavigation map with locations of remote operator triggers based onvarious existing data, such as: human-supervised autonomous vehicle testdata; operating data recorded by autonomous vehicles while operatingautonomously; accident data from human-operated vehicles; and/orcharacteristics of roads or intersections flagged for manual control.While autonomously navigating a planned route, an autonomous vehicle canexecute other Blocks of the method to: automatically request remoteoperator assistance as the autonomous vehicle approaches a location of aremote operator trigger indicated in the navigation map; automaticallycede decision-making or full operational control of the autonomousvehicle to a remote human operator; execute navigational commandsreceived from the remote human operator to navigate through thislocation; and then resume autonomous operation upon passing thislocation or upon confirmation from the remote human operator to resumeautonomous operation.

In particular, the remote computer system can access various historicaldata, such as: locations over which local human operators occupyingautonomous vehicles took manual control of their autonomous vehicles(e.g., during autonomous vehicle testing); locations at which autonomousvehicles, operating autonomously, unexpectedly disengaged (e.g., due toan autonomous operation failure or inability to verify a nextnavigational action); and/or locations (and severity, cost) of accidentsinvolving human-operated vehicles; etc. within a geographic region.Based on these historical data, the remote computer system can isolatediscrete locations, intersections, lanes, and/or other road segments atwhich an autonomous vehicle may be at greater risk for collision withother vehicles, may be delayed in executing a next navigational action,or may execute a next navigational action with reduced confidence. Theremote computer system can then populate a navigation map (or alocalization map, a table, or other container) with remote operatortriggers and related trigger parameters at geospatial locations of theseflagged road segments.

For example, the remote computer system can: generate a heatmap offrequencies of manual control selections, autonomous vehicledisengagements, and/or traffic accidents per instance of traversal by avehicle throughout the geographic region over a period of time; identifydiscrete geospatial locations or small geospatial areas within theheatmap exhibiting greatest frequencies of manual control selections,autonomous vehicle disengagements, and/or traffic accidents per instanceof traversal by a vehicle; write a remote operator flag to thenavigation map at each of these discrete geospatial locations or smallgeospatial areas; and push this navigation map (or a navigation mapupdate) to each autonomous vehicle deployed to this geographic region.In this example, the remote computer system can also derive correlationsbetween local conditions and these instances of manual controlselections, autonomous vehicle disengagements, and/or trafficaccidents—such as: time of day; local weather conditions; and anautonomous vehicle entering uncommon (e.g., five-way) intersections,entering railroad crossings, facing into the Sun, entering a schoolzone, nearing a large crowd of pedestrians, or approaching anunprotected left turn; etc. The remote computer system can then writethese conditions to corresponding remote operator triggers in thenavigation map (or localization map, table, or other container) in theform of trigger parameters.

During autonomous operation, an autonomous vehicle can: reference alocalization map to determine its geospatial location; and reference thenavigation map to elect and then execute navigational actions, such asaccelerating, braking, turning, changing lanes, etc. along a plannedroute toward a specified destination. As the autonomous vehicleapproaches a location of a remote operator trigger indicated by thenavigation map (or in the localization map, table, or other container),the autonomous vehicle can automatically transmit a request for manualassistance to a remote operator (or to a remote operator manager moregenerally). Once a remote operator is assigned to assist the autonomousvehicle in navigating through this location, the autonomous vehicle cantransition from autonomous navigation to remote manual control by theremote operator and can transmit (or “stream”) video, LIDAR, and/orother sensor data to the remote operator portal associated with theremote operator in real-time. The remote operator can view these sensordata through her remote operator portal and elect to: delay anavigational action (e.g., in the autonomous vehicle's queue); confirm anavigational action; select from a predefined set of navigationalactions; or manually adjust brake, accelerator, and/or steeringpositions accordingly. The autonomous vehicle can then transition backto full autonomous operation and resume full autonomous navigation alongthe planned route, such as: once the autonomous vehicle has moved pastthe location (or intersection, lane, and/or other road segment) linkedto this remote operator trigger; or once the remote operator hasconfirmed—via the remote operator portal—transition back to autonomousoperation.

For example, emergency scenario or accident data for training anautonomous vehicle solution may not be immediately available withoutinvolving autonomous vehicles (or vehicles outfitted with similar sensorsuites) in a variety of different accidents while collecting sensor datafrom these autonomous vehicles. Therefore, an autonomous vehiclesolution may not be trained to detect and respond to possible emergencyscenarios or to detect and respond to emergency scenarios in which it isdirectly involved, such as: occupying a railroad crossing as a trainapproaches; navigating past a vehicle that has crossed into oncomingtraffic near the autonomous vehicle; or approaching a large animalcrossing a road ahead of the autonomous vehicle. In order topreemptively handle the possibility of such emergency scenariosthroughout a geographic region, the remote computer system can: identifydiscrete locations, intersections, lanes, or other road segments atwhich emergency scenarios are particularly likely to occur (e.g.,locations associated with transition to manual control by local humanoperators while occupying these autonomous vehicles, locationsassociated with accident frequencies that substantially exceed athreshold, average, or baseline value); and then annotate a navigationmap or other container with remote operator triggers at correspondinglocations. An autonomous vehicle approaching a location associated witha remote operator trigger can automatically and preemptively requestassistance from a remote operator and serve sensor data to this remoteoperator prior to (e.g., ten seconds before) the autonomous vehicle'sarrival at this flagged location, thereby enabling the remote operatorto quickly perceive the scene around the autonomous vehicle and reliablyassume manual control of the autonomous vehicle prior to the autonomousvehicle executing a higher-risk navigational action or disengaging dueto a failure at the flagged location. A remote operator manager can alsodynamically and predictively allocate remote human operators to assistautonomous vehicles approaching locations of remote operator triggersindicated in the navigation map as these autonomous vehicles operate(e.g., execute routes) within a geographic region. Altogether, theremote computer system, remote operator portal, and fleet of autonomousvehicles can cooperate to annotate a navigation map with locations ofremote operator triggers and to implement this navigation map in orderto reduce risk to autonomous vehicles entering known higher-riskscenarios and in order to maintain high operating efficiency for theseautonomous vehicles.

In particular, the remote computer system can preemptively identifyhigher-risk road segments, road segments in which autonomous vehiclesmay be unable to detect and avoid risk, or road segments in whichautonomous vehicles may be unable to confidently elect a nextnavigational action and to label a navigational map (or other container)with remote operator triggers at corresponding locations. An autonomousvehicle (or the remote computer system) can then automatically trigger aremote operator to assume control of the autonomous vehicle and toassist navigation of the autonomous vehicle as the autonomous vehicleapproaches a road segment linked to a remote operator trigger in thenavigation map in order to: reduce risk of collision with other vehiclesor obstacles nearby; and/or maintain a high operating efficiency of theautonomous vehicle.

3. AUTONOMOUS VEHICLE AND SENSOR SUITE

Block S110 of the method recites, during a scan cycle, recordingmulti-dimensional sensor images at multi-dimensional sensors arranged onthe vehicle. Generally, in Block S110, an autonomous vehicle accessessensor data from various sensors arranged on or integrated in theautonomous vehicle—such as distance scans from multiple LIDAR sensorsand/or color 2D images from multiple color cameras—recordedapproximately concurrently by sensors defining fields of view exhibitingsome overlap over a distance range from the autonomous vehicle.

In one implementation, the autonomous vehicle includes: a suite ofsensors configured to collect information about the autonomous vehicle'senvironment; local memory that stores a navigation map defining laneconnections and nominal vehicle paths for a road area and a localizationmap that the autonomous vehicle implements to determine its location inreal space; and a controller that governs actuators within theautonomous vehicle to execute various functions based on the navigationmap, the localization map, and outputs of these sensors. In oneimplementation, the autonomous vehicle includes a set of 360° LIDARsensors arranged on the autonomous vehicle, such as one LIDAR sensormounted at each corner of the autonomous vehicle or a set of LIDARsensors integrated into a roof rack mounted to the roof of theautonomous vehicle. Each LIDAR sensor can output one three-dimensionaldistance scan—such as in the form of a 3D point cloud representingdistances between the LIDAR sensor and external surfaces within thefield of view of the LIDAR sensor—per rotation of the LIDAR sensor(i.e., once per scan cycle).

The autonomous vehicle can also be outfitted (or retrofit) withadditional sensors, such as: color cameras; 3D color cameras; auni-dimensional or multi-dimensional (e.g., scanning) RADAR or infrareddistance sensor; etc. The autonomous vehicle can implement similarmethods and techniques to read data from these sensors.

The autonomous vehicle can then: identify (or “perceive”) mutableobjects nearby from these sensor data; regularly compare these data tofeatures represented in a localization map in order to determine itslocation and orientation in real space; and identify a lane occupied bythe autonomous vehicle, a local speed limit, a next navigational action,and/or proximity of a remote operator trigger location, etc. based onthe autonomous vehicle's location and orientation and data stored in anavigation map. By regularly implementing these methods and techniquesin conjunction with a planned route, the autonomous vehicle canautonomously navigate toward a destination location in Block S140.

4. REMOTE OPERATOR TRIGGER LOCATIONS BY AUTONOMOUS VEHICLE TEST DATA

Block S110 of the method recites accessing a corpus of driving recordsof a fleet of autonomous vehicles operating within a geographic region;Block S120 of the method recites identifying a road segment, within thegeographic region, associated with a frequency of transitions, fromautonomous operation to local manual operation triggered by localoperators occupying autonomous vehicles in the fleet, that exceeds athreshold frequency based on the corpus of driving records; and BlockS130 of the method recites associating a location of the road segment,represented in a navigation map, with a remote operator trigger.Generally, in Blocks S110, S120, and S130, the remote computer systemcan: access operational data collected from autonomous vehicles occupiedby local human operators (e.g., “test drivers”) during autonomousvehicle test periods on public roads; extract manual operatortrends—such as location and or characteristics of adjacent road segmentsat times of manually-triggered transition from autonomous operation tomanual operation—from these operational data; and then define remoteoperator triggers at road segments associated with higher frequencies ofmanually-triggered transition (and at locations exhibiting similaritiesto road segments associated with higher frequencies ofmanually-triggered transition), as shown in FIGS. 1 and 3A.

An autonomous vehicle solution may be tested on public roads, such asover hundreds, thousands, or millions of miles. A human operatoroccupying an autonomous vehicle during a test period may manuallytransition the autonomous vehicle from autonomous operation (e.g., an“autonomous mode”) to manual operation (e.g., a “manual mode”), such aswhen the autonomous vehicle approaches a difficult intersection or inthe presence of an unexpected obstacle (e.g., a vehicle, a pedestrian,an animal) near or in the path of the autonomous vehicle. The autonomousvehicle (and/or the remote computer system) can record characteristicsof such instances of human-triggered transitions to manual control, suchas including: locations; times of day; local traffic conditions;constellations of detected obstacles nearby; lanes occupied byautonomous vehicles; road characteristics (e.g., road surface quality,wetness, color, reflectivity); weather conditions; and/or position ofthe autonomous vehicle relative to the Sun, Sun intensity, or sensorobscuration due to sunlight; etc. at (and slightly before) local humanoperators triggered these autonomous-to-manual-operation transitions.The remote computer system can then aggregate these data in a remotedatabase over time.

The remote computer system can then analyze theseautonomous-to-manual-operation transitions and related data to isolateroad segments and local conditions likely to necessitate remote manualcontrol. In particular, a (significant) proportion of theseautonomous-to-manual-operation transitions may be arbitrary (e.g.,anomalous, haphazard). However, locations, times of day, local trafficconditions, and/or other conditions of some of theseautonomous-to-manual-operation transitions may repeat with relativelyhigh frequency over time. The remote computer system can therefore:aggregate locations of these autonomous-to-manual-operation transitionsoccurring during road test periods throughout a geographic region overtime; and identify road segments over which local human operatorscommonly transition their autonomous vehicles from autonomous operationto manual control in Block S120, such as with greater absolutefrequency, greater frequency per instance the road segment is traversed,or greater frequency per unit time.

4.1 EXAMPLE: GEOSPATIAL PROXIMITY

For example, in Block S110, the remote computer system can accessgeospatial locations of instances of transition from autonomousoperation to local manual operation triggered by local operatorsoccupying autonomous vehicles in a fleet of autonomous vehicles overtime (e.g., during test periods within a geographic region prior todeployment of this fleet of autonomous vehicles for full autonomousoperation within this geographic region). The remote computer system canthen aggregate instances of transition from autonomous operation tomanual operation across this fleet of autonomous vehicles over time intoa set of groups based on geospatial proximity of these transitions. Foreach group in this set of groups, the remote computer system can:calculate a frequency of autonomous-to-manual-operation transitionsalong a particular road segment—containing geospatial locations ofautonomous-to-manual-operation transitions in this group—such as basedon a ratio of total quantity of transitions in the first group toquantity of instances of autonomous vehicles in the fleet traversingthis road segment; and then flag this road segment if this frequency oftransitions exceeds a threshold frequency (e.g., 30%) in Block S120. Theremote computer system can then write a remote operator trigger to eachof these flagged road segments in a navigation map for this geographicregion.

4.2 EXAMPLE: TEMPORAL PROXIMITY

Furthermore, in the foregoing example, the remote computer system can:access times of these instances of transition from autonomous operationto manual operation; and aggregate these autonomous-to-manual-operationtransitions into groups further based on temporal proximity (e.g.,occurring during the same day of the week and/or during similar times ofday). For each group in this set, the remote computer system can: flag aroad segment—containing geospatial locations ofautonomous-to-manual-operation transitions in this group—if thefrequency of transitions along this road segment within a time windowrepresented in this group exceeds a threshold frequency in Block S120;and then write a remote operator trigger with a constraint of this timewindow to this flagged road segment in the navigation map in Block S130.In this example, the remote computer system can thus limit a remoteoperator trigger for a road segment flagged in the navigation mapaccording to a time window; and the autonomous vehicle can transmit arequest for manual assistance to a remote operator only upon approachingthis road segment during the time window defined in this remote operatortrigger.

4.3 EXAMPLE: SCENE CHARACTERISTICS

In a similar example shown in FIG. 3A, the remote computer systemaccesses both: geospatial locations of autonomous-to-manual-operationtransitions triggered by local operators occupying autonomous vehiclesin the fleet; and scene characteristics (e.g., local traffic conditions,constellations of obstacles nearby, road surface quality, road wetness,road color, road reflectivity, local weather conditions) proximalautonomous vehicles during these autonomous-to-manual-operationtransitions in Block S110. The remote computer system then aggregatesautonomous-to-manual-operation transitions into a set of groups based onboth geospatial proximity and similarity of scene characteristicsproximal autonomous vehicles during these transitions. For each group inthis set, the remote computer system can: flag a road segment—containinggeospatial locations of autonomous-to-manual-operation transitions inthis group—if the frequency of transitions occurring along this roadsegment concurrently with a particular scene characteristicrepresentative of this group exceeds a threshold frequency in BlockS120; and then write a remote operator trigger with a constraint of thisparticular scene characteristic to this flagged road segment in thenavigation map in Block S130. In this example, the remote computersystem can thus limit a remote operator trigger for a road segmentflagged in the navigation map according to a scene characteristic (or aconstellation of scene characteristics); and the autonomous vehicle cantransmit a request for manual assistance to a remote operator only upondetecting this scene characteristic (or a constellation of scenecharacteristics) when approaching this road segment.

4.4 EXAMPLE: OBFUSCATION BY SOLAR RADIATION

In a similar example, the remote computer system can: access offsetsbetween anteroposterior axes of autonomous vehicles and the Sun duringautonomous-to-manual-operation transitions in Block S110; identify agroup of autonomous-to-manual-operation transitions occurring atgeospatial locations along a road segment concurrent with solaroffsets—between anteroposterior axes of autonomous vehicles and theSun—that fall within a solar offset window in Block S120; write a remoteoperator trigger to this road segment in the navigation map if thisfrequency of autonomous-to-manual-operation transitions in this groupexceeds a threshold frequency in Block S130; and then limit this remoteoperator trigger according to this solar offset window in Block S130.The remote computer system can similarly calculate this solar offsetwindow based on positions of autonomous vehicles relative to the Sunwhen solar radiation overwhelmed sensors (e.g., color cameras, LIDARsensors) in these autonomous vehicles, such as along this road segment,and associate a remote operator trigger and solar offset window withthis road segment accordingly. Later, as an autonomous vehicleapproaches this road segment, the autonomous vehicle can transmit arequest for manual assistance to a remote operator if the offset betweenan anteroposterior axes of the autonomous vehicle and the Sun fallswithin this solar offset window.

4.5 EXAMPLE: PEDESTRIANS

In a similar example, the remote computer system can: detect presence(e.g., quantities) of pedestrians proximal autonomous vehicles duringautonomous-to-manual-operation transitions in Block S110; identify agroup of autonomous-to-manual-operation transitions occurring atgeospatial locations along a road segment concurrent with presence of aminimum quantity (or a range) of pedestrians in Block S120; write aremote operator trigger to this road segment in the navigation map ifthis frequency of autonomous-to-manual-operation transitions in thisgroup exceeds a threshold frequency in Block S130; and then limit thisremote operator trigger according to this minimum quantity (or a range)of pedestrians in Block S130. Later, as an autonomous vehicle approachesthis road segment, the autonomous vehicle can transmit a request formanual assistance to a remote operator if the autonomous vehicle hasdetected at least the minimum quantity of pedestrians in its vicinity.

4.6 EXAMPLE: HEATMAP AND DYNAMIC REMOTE OPERATOR TRIGGERS

In the foregoing examples, the remote computer system can generate aheatmap of autonomous-to-manual-operation transitions during autonomousvehicle test periods throughout a geographic region, such as with groupsof transitions weighted by spatial density and by ratio of number oftransitions to total autonomous vehicle traversals across road segmentsin this geographic region. The remote computer system can then rank roadsegments in this geographic region by intensity in the heatmap. When afleet of autonomous vehicles is deployed to operate autonomously in thegeographic region, the remote computer system (or autonomous vehicles)can dynamically set and clear remote operator triggers at road segmentswithin this geographic region based on rank of these road segments andavailability of remote operators to handle remote operator requests fromthese autonomous vehicles. In particular, the remote computer system canimplement load balancing techniques to activate remote operator triggersfor highest-ranking road segments and to selectively activate remoteoperator triggers for lower-ranking road segments responsive toincreased availability of remote operators to respond to remote operatorrequests from these autonomous vehicles.

4.7 REMOTE OPERATOR TRIGGER GENERATION

The remote computer system can then selectively annotate a navigationmap with remote operator triggers in Block S130. For example, the remotecomputer system can annotate the navigation map with remote operatortriggers at discrete locations, intersections, lanes, and/or segments ofroadway at which local human operators in these autonomous vehiclesfrequently elected manual control. For example, the remote computersystem can annotate the navigation map with a remote operator triggerfor each discrete road segment and vehicle direction: over which localoperators elected manual control of their autonomous vehicles more thana threshold number of times per instance that an autonomous vehicletraversed this road segment; or over which local operators electedmanual control of their autonomous vehicles more than a threshold numberof times per unit time; etc.

However, the autonomous vehicle can implement any other methods ortechniques to extract manual control locations from historicalautonomous vehicle test data and to automatically annotate thenavigation map with remote operator triggers.

5. REMOTE OPERATOR TRIGGERS BY AUTONOMOUS VEHICLE DISENGAGEMENTS

In one variation shown in FIG. 3B, the remote computer system definesremote operator triggers based on historical autonomous vehicledisengagements—that is, instances in which autonomous vehicles in thefleet automatically ceased autonomous operation, such as due to failureof an autonomous technology, inability to perceive their surroundingswith sufficient confidence, or inability to verify next navigationalactions.

In this variation, the remote computer system can: identify a roadsegment associated with a frequency of autonomous-to-manual-operationtransitions—triggered by autonomous vehicles, rather than by local humanoperators occupying these autonomous vehicles—that exceeds a thresholdfrequency based on the corpus of driving records accessed in Block S110;and then associate a location of this road segment, represented in thenavigation map, with a remote operator trigger in Block S130. In thisvariation, the remote computer system can also implement methods andtechniques described above to associate this remote operator triggerwith addition conditions.

6. REMOTE OPERATOR TRIGGERS BY HISTORICAL ACCIDENT DATA

In one variation shown in FIG. 3C: Block S110 of the method includesaccessing historical accident data of human-operated vehicles involvedin road accidents within a geographic region; and Block S120 of themethod includes identifying a road segment, within the geographicregion, associated with a frequency of accidents exceeding a thresholdaccident frequency. Generally, in this variation, the remote computersystem can: access road vehicle accident data, such as from an accidentdatabase for human-operated vehicles, in Block S110; and then extracttrends from these data to identify locations (and local conditions) forwhich greater risk of accidents or collisions exist in Block S120. Theremote computer system can then define remote operator flags for theselocations (and conditions) and write these remote operator flags to thenavigation map (or other container) accordingly in Block S130, as shownin FIGS. 1 and 3C.

In one implementation, the remote computer system extracts, fromavailable accident data: geospatial locations (e.g., latitudes andlongitudes); lane identifiers; and directions of motion of vehiclesinvolved in recorded accidents. The remote computer system can alsoextract, from these accident data: navigational actions; days; times ofday; weather conditions; numbers and types of vehicles involved (e.g.,cars, trucks, cyclists); numbers of pedestrians involved; accidentseverities (e.g., minor impact, vehicle totaled); types of accidents(e.g., rear-end collisions, side-impact collisions, sideswipecollisions, vehicle rollover, head-on collisions, or multi-vehiclepile-ups); etc. of recorded accidents and vehicles involved in theseaccidents.

The remote computer system can then compile these accident data into ageospatial heatmap of accidents. For example, the remote computer systemcan weight each incidence of a recorded accident by: how recently theaccident occurred; a number of vehicles involved in the accident; and/ora severity of the accident (e.g., as a function of total cost of vehicledamage and human injuries). The remote computer system can then flagdiscrete geospatial locations, specific intersections, or specific roadsegments (e.g., a 100-meter lengths of road) over which weighted ratesof accidents per unit vehicle passing this location or per unit timeexceed a threshold rate. (In this example, the remote computer systemcan adjust the threshold rate as a function of availability of remoteoperators.)

Upon amassing a set of discrete geospatial locations, specificintersections, and/or specific road segments at which accidents haveoccurred in the past, such as with significant frequency and/orseverity, the remote computer system can write remote operator triggersto these locations, intersections, and/or road segments in thenavigation map in Block S130, as described above.

The remote computer system can also write a weight (or “priority”) valueto each of these remote operator triggers; and the autonomous vehicleand the remote computer system can cooperate to selectively engage aremote operator to assist the autonomous vehicle in passing a locationof a remote operator trigger based on a weight assigned to this remoteoperator trigger and current resource load at the remote operatormanager (i.e., based on current availability of remote operators), asdescribed below.

Therefore, the remote computer system can: access a corpus of historicaltraffic accident data of manually-operated vehicles involved in trafficaccidents within a geographic region in Block S110; identify a roadsegment—within this geographic region—associated with a frequency oftraffic accidents that exceeds a threshold frequency in Block S120 basedon this corpus of historical traffic accident data; and associate alocation of this road segment with a remote operator trigger accordinglyin Block S130.

7. REMOTE OPERATOR TRIGGERS BY ROAD CHARACTERISTICS

In another variation, Block S110 of the method recites accessing aspecification for triggering manual control of autonomous vehicles; andBlock S120 of the method recites identifying a road segment, within ageographic region, exhibiting characteristics defined by thespecification. Generally, in this variation, the remote computer systemcan: access a remote operator trigger specification defined by a humanor generate a remote operator trigger specification based on historicalautonomous vehicle operation and remote operator data in Block S110;then scan a navigational map, autonomous vehicle data, and/or existingtraffic data, etc. for a geographic region for locations associated withcharacteristics that match the remote operator trigger specification;and flag these locations for assignment of remote operator triggers.

In one implementation shown in FIG. 1, the remote computer system:accesses a manual list of characteristics of locations of remoteoperator triggers or automatically characterizes these locations basedon available test period and/or accident data in Block S110; and thenscans a navigation map for discrete locations, intersections, and/orroad segments that exhibit similar characteristics in Block S120. Inthis implementation, the remote computer system can automaticallypopulate the navigation map with remote operator triggers based oncharacteristics of roadways represented in the navigation map ratherthan specifically as a function of past manual control and/or accidentlocations.

In another implementation, the remote computer system further processesmanual control data for autonomous vehicle test periods—describedabove—and extracts additional trends from these data, such as:autonomous vehicle direction; lane occupied by an autonomous vehicle;navigational action (e.g., turning, lane change, merging) performedbefore, during, and/or after an autonomous-to-manual-operationtransition; times of day; local traffic conditions (e.g., vehicletraffic density and speed); lengths of road segments traversed duringautonomous-to-manual-operation transitions; types and proximity ofobstacles near an autonomous vehicle during anautonomous-to-manual-operation transition; etc. Based on these trends,the remote computer system can correlate various parameters—such asnavigational action, intersection type, road segment type, etc.—toelected manual control of autonomous vehicles. In particular, the remotecomputer system can implement pattern recognition, regression, or othertechniques to correlate local operator manual control of autonomousvehicles to certain characteristics of intersections or road segments.For example, the remote computer system can identify discrete lanesegments and navigational actions over which local human operators arelikely to elect manual control of autonomous vehicles, such as: rightturns exceeding 110°; navigating through railroad crossings; navigatingthrough road construction; unprotected left turns; etc. The remotecomputer system can then: scan the navigation map for road segments orintersections, etc. that exhibit substantially similar characteristicsin Block S120; and annotate the navigation map with remote operatortriggers at these locations accordingly in Block S130, as describedabove.

The remote computer system can implement similar methods and techniquesto correlate accidents with certain characteristics of intersections orroad segments in Block S110 and then scan and annotate the navigationmap accordingly in Block S120 and S130.

7.1 EXAMPLES

In one example, the remote computer system correlates unprotected leftturns with above-average rates of manual control by local operatorsand/or above-average rates of accidents in Block S120. Accordingly, theremote computer system identifies all unprotected left turns representedin the navigational map and labels the corresponding locations as remoteoperator triggers in Block S130. The autonomous vehicle thus submits arequest for manual assistance in Block S150 upon approaching anunprotected left turn.

In the foregoing example, the remote computer system can also identify acorrelation between unprotected left turns and manual control by localoperators and/or above-average rates of accidents during high-trafficperiods, when local traffic is moving at high speed, or duringparticular times of day. The remote computer system can then annotatethe navigation map with remote operator triggers—including temporal,traffic, and/or other local condition parameters—at locations of knownunprotected left turns represented in the navigation map in Block S130.Upon approaching an unprotected left turn at a time specified by acorresponding remote operator trigger in the navigation map or whenlocal traffic conditions match or exceed minimum traffic conditionsspecified by the remote operator trigger, the autonomous vehicle cansubmit a request for manual assistance in Block S150 and thenautomatically transition to manual control by the remote operator, suchas upon entering the corresponding unprotected left turn lane, in BlockS154. Upon completing the left turn under manual control or guidance,the autonomous vehicle can transition back to autonomous navigation.However, if the autonomous vehicle determines that conditions specifiedby the remote operator trigger have not been met—based on data collectedby the autonomous vehicle in real-time as the autonomous vehicleapproaches this unprotected left turn—the autonomous vehicle canautonomously navigate through the unprotected left turn with remoteoperator assistance.

In another example, the remote computer system annotates the navigationmap with remote operator triggers at locations of all known railroadcrossings. The computer vision can also write a conditionaltraffic-related statement to remote operator triggers for these knownrailroad crossings, such as confirmation to request remote operatorassistance if another vehicle is stopped in the autonomous vehicle'slane, on the other side of the railroad crossing, and within a thresholddistance of the railroad crossing (e.g., three car lengths or twentymeters).

7.2 REMOTE OPERATE TRIGGER PROPAGATION

In a similar implementation shown in FIGS. 3A and 3C, after assigning aremote operator trigger to a road segment in Block S130, the remotecomputer system can: derive a set of characteristics of the roadsegment; scanning the navigation map—of the geographic region containingthis road segment—for a second road segment exhibiting characteristicssimilar to those of the road segment; associate a second location of thesecond road segment with a second remote operator trigger; in BlockS130; and write this remote operator trigger to the navigation map (orother container).

The remote computer system can therefore automatically identifyadditional road segments—that may obligate remote manual operation overautonomous operation for deployed autonomous vehicles—in a geographicregion, even if historical data for autonomous vehicle operation throughthese road segments is unavailable or limited, based on similaritiesbetween these additional road segments and road segments previouslyassociated with remote operator triggers.

8. LOCATION-AGNOSTIC REMOTE OPERATOR TRIGGERS

In one variation, the remote computer system can derive a constellationof location-agnostic scene characteristics and/or autonomous vehiclecharacteristics exhibiting high correlation withautonomous-to-manual-operation transitions, autonomous vehicledisengagements, traffic accidents, etc. from historical data describedabove, such as by implementing regression or deep learning techniques.The remote computer system can then define remote operator triggers fordeployed autonomous vehicles based on this constellation oflocation-agnostic scenes and/or autonomous vehicle characteristics. Forexample, the remote computer system can define a constellation oflocation-agnostic scene characteristics and/or autonomous vehiclecharacteristics including: damp-to-wet road condition; solar offsetwindow (e.g., within the range of +/−15° zenith and +/−20° azimuthal tothe Sun); and pedestrian present. Thus, when an autonomous vehicleoperating in an autonomous mode detects a pedestrian and falls withinthis solar offset window during or after rainfall, the autonomousvehicle can serve a request to the remote operator manager for remoteoperator assistance according to this location-agnostic remote operatortrigger.

9. REMOTE OPERATOR REQUEST

Block S140 of the method recites, at an autonomous vehicle, autonomouslynavigating along a route; and Block S150 recites, at the autonomousvehicle, transmitting a request for manual assistance to the remoteoperator in response to approaching the location associated with theremote operator trigger. Generally, in Block S140, the autonomousvehicle can implement autonomous navigation techniques to autonomouslynavigate from a start location (e.g., a pickup location specified by arideshare user), along a route, toward a destination location (e.g., adropoff location specified by the rideshare user). While navigatingalong this route, the autonomous vehicle can monitor its location and/orcharacteristics of a scene around the autonomous vehicle for conditionspecified in a remote operator trigger, such as defined in a navigationmap stored locally on the autonomous vehicle. Thus, as the autonomousvehicle approaches a road segment specified in a remote operator trigger(and confirms scene and/or autonomous vehicle characteristics specifiedin this remote operator trigger, such as traffic, weather, and time ofday conditions), the autonomous vehicle can transmit a request for humanassistance to a remote operator. Accordingly, the autonomous vehicle cancede operational controls to a remote operator in Block S154 until theautonomous vehicle passes the road segment or until autonomous controlis returned to the autonomous vehicle by the remote operator. Inparticular, the autonomous vehicle can identify a set of conditions(e.g., autonomous vehicle location and orientation, local conditions)that fulfill a remote operator trigger and, accordingly, automaticallyreturn a request for manual human assistance to a remote operatormanager (or to a remote operator directly).

In one implementation, while autonomously navigating along a route thatintersects a location of a remote operator trigger defined in thenavigation map (or other container), the autonomous vehicle: estimatesits time of arrival at this location; and then transmits a request formanual assistance to the remote operator manager in response to thistime of arrival falling below a threshold duration (e.g., ten seconds;or five seconds when the autonomous vehicle is travelling at ten milesper hour, ten seconds when the autonomous vehicle is travelling atthirty miles per hour, and fifteen seconds when the autonomous vehicleis travelling at sixty miles per hour).

Alternatively, the remote computer system can define a remote operatortrigger along a length of road segment in Block S130; and the autonomousvehicle can automatically transmit a request for manual assistance tothe remote operator manager in Block S150 in response to entering thisroad segment. For example, the remote computer system can define ageoreferenced boundary around a cluster ofautonomous-to-manual-operation transitions and offset outwardly from aperimeter of this cluster by a trigger distance (e.g., 30 meters) andlink this georeferenced boundary to a remote operator trigger for a roadsegment in Block S130. Upon crossing this georeferenced boundary andentering this road segment (and upon confirming other conditionsspecified in the remote operator trigger), the autonomous vehicle canautomatically transmit a request for manual assistance to the remoteoperator manager in Block S150.

Upon receipt of a request for manual assistance from the autonomousvehicle, the remote operator manager can: select a particular remoteoperator from a set of available remote operators; and then route sensordata—received from the autonomous vehicle in Block S152 describedbelow—to a remote operator portal associated with the remote operator,such as via a computer network. For example, upon receipt of a requestfor manual assistance from the autonomous vehicle responsive to a remoteoperator trigger, the remote operator manager can selectively reject theautonomous vehicle's request or connect the autonomous vehicle to anavailable remote operator based on a weight or priority associated withthis remote operator trigger and based on current resource load (i.e.,availability or remote operators). In this example, the remote operatormanager can implement resource allocation techniques to assignautonomous vehicles approaching locations of highest-priority remoteoperator triggers to available remote operators first, then assignautonomous vehicles approaching locations of lower-priority remoteoperator triggers to available remote operators up until a targetresource load is met (e.g., 90% of remote operators are currentlyassisting autonomous vehicles).

Alternatively, the autonomous vehicle can serve a request for manualassistance directly to a remote operator. For example, a remote operatorcan be assigned a preselected set of autonomous vehicles currently inoperation with a geographic region, and the remote operator can monitorlow-resolution sensor data—streamed from these autonomous vehicles whenoperating within the geographic region—through her remote operatorportal. When the autonomous vehicle enters or approaches a road segmentassociated with a remote operator trigger, the autonomous vehicle canreturn a request for remote operator control and return high-resolutionsensor data (e.g., lower compression, larger sized, and/or greater framerate color camera data) directly to the remote operator's portal inBlock S150. Accordingly, the remote operator portal can surface a sensorfeed from the autonomous vehicle, enable remote controls for theautonomous vehicle, and prompt the remote operator to remotely engagethe autonomous vehicle.

Yet alternatively, the remote computer system can automatically: trackan autonomous vehicle; generate a request for manual assistance for theautonomous vehicle when conditions of a remote operator trigger are metat the autonomous vehicle; and serve this request to a remote operator.For example, the autonomous vehicle can stream low-resolution sensor,perception, and/or telemetry data to the remote computer systemthroughout operation. The remote computer system can then automaticallyqueue a remote operator to assume manual control of the autonomousvehicle when telemetry data received from the autonomous vehicleindicates that the autonomous vehicle is approaching a location assigneda remote operator trigger and when low-resolution perception data (e.g.,types and locations of objects detected in camera and/or LIDAR datarecorded by the autonomous vehicle) received from the autonomous vehicleindicates that conditions of this remote operator trigger are met.

However, the autonomous vehicle, the remote operator manager, and/or theremote computer system can implement any other method or technique toselectively connect the autonomous vehicle to a remote operatorresponsive to a request for manual assistance based on a remote operatortrigger.

10. REMOTE CONTROLS

The method further includes: Block S152, which recites transmittingsensor data to a remote operator portal associated with the remoteoperator; Block S154, which recites executing a navigational commandreceived from the remote operator via the remote operator portal; andBlock S160, which recites, in response to passing the location, resumingautonomous navigation along the planned route in Block S160. Generally,in Block S152, the autonomous vehicle can serve data—such as raw sensor,perception, and/or telemetry data sufficient for enabling a remoteoperator to efficiently and reliably trigger a navigational action orassume manual control of the autonomous vehicle—to a remote operator.The autonomous vehicle can then: execute commands received from theremote operator in Block S154 in order to navigate through or past theroad segment linked to the remote operator trigger; and transition backto autonomous operation in Block S160 upon exiting this road segmentand/or upon confirmation from the remote operator to resume autonomousnavigation, as shown in FIGS. 2 and 4.

For example, once the autonomous vehicle determines that conditions ofthe remote operator trigger are met, returns a request for manualassistance to the remote operator, and/or receives confirmation ofmanual assistance from the remote operator or remote operator manager atan initial remote operation time, the autonomous vehicle can stream rawsensor data, perception data (e.g., perception of a scene around theautonomous vehicle derived from raw sensor data recorded through sensorsin the autonomous vehicle), and/or telemetry data to the remote operatorportal in real-time over a wireless computer network following theinitial remote operation time. The autonomous vehicle can concurrentlytransition control of some or all actuators in the autonomous vehicle tothe remote operator portal.

10.1 BINARY REMOTE CONTROL FUNCTION

In one implementation shown in FIGS. 1 and 2, as the autonomous vehicleapproaches the location of a remote operator trigger and once a remoteoperator is assigned to the autonomous vehicle, the autonomous vehicle(or the remote operator manager) enables a binary control function ofthe autonomous vehicle at the remote operator's portal, such asincluding: a confirm function to trigger the autonomous vehicle toexecute a preselected navigational action (e.g., enter an intersectionor execute a left turn through the road segment associated with theremote operator trigger); and a delay function to delay execution ofthis preselected navigational action.

In one example, the remote computer system: writes a remote operatortrigger to an unprotected left turn in the navigation map; and assigns abinary control function—including navigational action confirmation anddelay options—to this remote operator trigger in Block S130. As theautonomous vehicle traverses its assigned route and approaches thisunprotected left turn in Block S140, the autonomous vehicle can: querythe remote operator manager for remote manual control according to thesebinary control functions in Block S150. Once the remote operator managerassigns a remote operator to the autonomous vehicle, the autonomousvehicle can stream sensor data to the remote operator manager in BlockS152, such as: color camera feeds from forward-, left-, and right-facingcameras on the autonomous vehicle; composite point clouds containingconcurrent data output by multiple LIDAR sensors on the autonomousvehicle; telemetry data; and/or vehicle speed, braking position, andaccelerator position data; etc. The remote operator manager can thendistribute these sensor feeds to the operator portal associated withthis remote operator.

In this example, as the autonomous vehicle nears the unprotected leftturn, the autonomous vehicle can autonomously slow to a stop just aheadof this intersection while awaiting a command from the remote operator.Simultaneously, the remote operator portal can render these sensor datafor the remote operator in (near) real-time and enable binary controlsfor the autonomous vehicle. Upon determining that the autonomous vehiclehas right of way to enter the intersection ahead and will avoid oncomingtraffic when executing a left turn action, the remote operator cansubmit confirmation to execute the planned left turn action; uponreceipt of confirmation to execute the planned left turn action, theautonomous vehicle can resume autonomous execution of its planned route,including entering the intersection ahead and autonomously executing theleft turn, in Blocks S154 and S160.

Therefore, in this example, the autonomous vehicle can: slow to a stopin response to approaching a location associated with a remote operatortrigger; transmit a request for manual confirmation to resume autonomousnavigation along the route as the autonomous vehicle slows upon approachto this location; and then resume autonomous navigation along thisroute—past the location specified by the remote operator trigger—inresponse to receipt of binary, manual confirmation from the remoteoperator.

10.2 MULTIPLE REMOTE CONTROL FUNCTIONS

In a similar implementation, the remote computer system can assignmultiple possible navigational actions—such as “delay,” “sharp left,”“sweeping left,” “slow left,” and/or “fast left”—to a remote operatortrigger in Block S130. As an autonomous vehicle approaches the locationspecified by this remote operator trigger, the autonomous vehicle cantransmit a request to the remote operator manager for manual assistance;and the remote operator manager can assign the autonomous vehicle to aremote operator and enable selection of navigational actions specifiedby this remote operator trigger at this remote operator's portal. Theautonomous vehicle can then execute navigational actions selected by theremote operator via the remote operator portal in Block S154.

Once the remote operator confirms one or a subset of these navigationalactions, once the autonomous vehicle has moved fully past the locationspecified in this remote operator trigger, and/or once the remoteoperator confirms transition back to autonomous operation, theautonomous vehicle can return to full autonomous operation in BlockS160.

10.3 FULL REMOTE CONTROL FUNCTIONS

In another implementation, the remote computer system assigns fullmanual control of an autonomous vehicle—such as including control ofbrake, accelerator, and steering actuators in the autonomous vehicle—toa remote operator trigger in Block S130. Thus, as an autonomous vehicleapproaches the location specified in this remote operator trigger whileautonomously navigating along a planned route, the autonomous vehiclecan request assistance from a remote operator in Block S150. Once theremote operator manager assigns the autonomous vehicle to a remoteoperator, the autonomous vehicle, the remote computer system, and theremote operator's portal can cooperate to transition real-timedrive-by-wire controls of brake, accelerator, and steering positions inthe autonomous vehicle to the remote operator portal

For example, once the remote operator is assigned to assist theautonomous vehicle: the autonomous vehicle can stream sensor data to theremote operator manager for distribution to the remote operator portalin Block S152; and the autonomous vehicle and the computer system cancooperate to transition from 100% autonomous/0% manual control ofactuators in the autonomous vehicle to 0% autonomous/100% manual controlof these actuators over a period of time (e.g., four seconds). Theremote operator can thus assume full manual control of the autonomousvehicle—such as via a joystick or other interface connected to theremote operator portal—and remotely navigate the autonomous vehiclethrough the location or road segment associated with this remoteoperator trigger.

Furthermore, once the autonomous vehicle is fully past the location orroad segment linked to this remote operator trigger—such as confirmed bythe remote operator—the autonomous vehicle and the remote computersystem can cooperate to transition from 0% autonomous/100% manualcontrol back to 100% autonomous/0% manual control, such asinstantaneously or over a period of time (e.g., two seconds) in BlockS160.

Therefore, in response to confirmation of manual assistance from theremote operator, the autonomous vehicle can transfer braking,acceleration, and steering controls of the autonomous vehicle to theremote operator portal; and then execute braking, acceleration, and/orsteering commands received from the remote operator portal in BlockS154. The autonomous vehicle can then: cease transmission of sensor datato the remote operator portal and resume autonomous navigation along itsassigned route after passing the location and/or in response to receiptof confirmation from the remote operator to resume autonomous navigationin Block S160.

11. DAISY CHAIN

In one variation, the remote computer system (or the remote operatorportal) identifies multiple autonomous vehicles scheduled or anticipatedto approach a location of a remote operator trigger within a shortperiod of time and then assigns a single remote operator to manuallyassist each of these autonomous vehicles as they sequentially traversethis location. For example, the remote computer system can group astring of five autonomous vehicles (out of eight total vehicles) in-lineat an unprotected left turn and enable manual control of these vehiclesto a single remote operator; the remote operator can then manuallyconfirm execution of a left turn action for each autonomous vehicle inthe group individually or for the group of autonomous vehicles as awhole.

Therefore, in this variation, because a remote operator may becomeincreasingly familiar with a segment of road, an intersection, currenttraffic conditions, current weather conditions, current pedestriantraffic, etc. near a location or road segment linked to a remoteoperator trigger as the remote operator handles manual assistancerequests from autonomous vehicles passing this location or road segmentover time, the remote computer system can reduce cognitive load on theremote operator by continuing to assign autonomous vehicles approachingthis location or road segment to this same remote operator, such aswithin a short, contiguous duration of time. In particular, the remotecomputer system can assign the same remote operator to multipleautonomous vehicles passing through a particular remote operator triggerlocation within a limited period of time in order to enable the remoteoperator to make rapid, higher-accuracy navigational decisions for theseautonomous vehicles and with less cognitive load.

Once a number of autonomous vehicles approaching this remote operatortrigger location drops below a preset threshold quantity or frequency,the remote computer system can transition the remote operator to assistother autonomous vehicles passing or approaching other locations or roadsegments in the geographic region associated with remote operatortriggers.

Alternatively, the remote computer system can dedicate a particularremote operator trigger to a single remote operator, such as over thefull duration of this remote operator's shift. Therefore, in thisvariation, the remote computer system can assign this particular remoteoperator to assist each autonomous vehicle that approaches the locationspecified by this remote operator trigger over this period of time.

However, the remote computer system can cooperate with an autonomousvehicle in any other way to selectively and intermittently enable manualcontrol of the autonomous vehicle by a remote operator as the autonomousvehicle approaches and navigates past a remote operator trigger locationdefined in a navigation map.

12. AUTONOMOUS TECHNOLOGY UPDATE

In one variation, the remote computer system (or autonomous vehicles inthe fleet) can aggregate: sensor data (e.g., camera, LIDAR, andtelemetry data) recorded by autonomous vehicles when approaching,entering, and passing locations or road segments specified by remoteoperator triggers; remote operator commands returned to these autonomousvehicles while responding to autonomous vehicle requests for manualassistance; and results of execution of these commands by theseautonomous vehicles (e.g., whether an autonomous vehicle collided withanother object, proximity of other objects to the autonomous vehicleduring execution of navigational commands received from a remoteoperator). The remote computer system can then implement deep learning,artificial intelligence, regression, and/or other methods and techniquesto refine an autonomous navigation model based on these data. Inparticular, the remote computer system can implement deep learning,artificial intelligence, or similar techniques to retrain an autonomousnavigation (or “path planning”) model based on sensor data, remoteoperator commands, and navigation results recorded near locations ofremote operator triggers. For example, the autonomous vehicle canretrain the autonomous navigation model to elect a navigational action,an autonomous vehicle trajectory, and/or autonomous vehicle actuatorpositions more rapidly and/or more accurately at these remote operatortrigger locations based on scene and autonomous vehicle characteristicsnear these locations and navigational commands issued by remoteoperators when guiding these autonomous vehicles through these remoteoperator trigger locations. The remote computer system can then pushthis retrained (or “updated,” “revised”) autonomous navigation model todeployed autonomous vehicles, which can then implement this autonomousnavigation model when operating autonomously, thereby reducing need forremote operators to manually assist these autonomous vehicles nearremote operator trigger locations.

For example, as the remote computer system updates the autonomousnavigation model as described above and pushes updated autonomousnavigation models to deployed autonomous vehicles over time, the remotecomputer system can transition remote operator triggers from full remotemanual control to binary control—as described above—given thatautonomous vehicles executing updated autonomous navigation models maybe increasingly better suited to quickly and accurately select nextnavigational actions when approaching these remote operator triggerlocations. By thus transitioning these remote operator triggers fromfull remote manual control to binary control, the remote computer systemcan reduce involvement and resource load of remote operators tasked withremotely assisting these autonomous vehicles over time.

Therefore, the remote computer system can: record a corpus of sensordata received from an autonomous vehicle following a request for manualassistance as the autonomous vehicle approaches a remote operatortrigger location; record a navigational command entered by a remoteoperator assigned to this autonomous vehicle and served to theautonomous vehicle responsive to this request for manual assistance;generate a revised autonomous navigation model based on this corpus ofsensor data and the navigational command; and load this revisedautonomous navigation model onto the autonomous vehicle—and otherautonomous vehicles in the fleet.

The remote computer systems and methods described herein can be embodiedand/or implemented at least in part as a machine configured to receive acomputer-readable medium storing computer-readable instructions. Theinstructions can be executed by computer-executable componentsintegrated with the application, applet, host, server, network, website,communication service, communication interface,hardware/firmware/software elements of a human annotator computer ormobile device, wristband, smartphone, or any suitable combinationthereof. Other systems and methods of the embodiment can be embodiedand/or implemented at least in part as a machine configured to receive acomputer-readable medium storing computer-readable instructions. Theinstructions can be executed by computer-executable componentsintegrated by computer-executable components integrated with apparatusesand networks of the type described above. The computer-readable mediumcan be stored on any suitable computer readable media such as RAMs,ROMs, flash memory, EEPROMs, optical devices (CD or DVD), hard drives,floppy drives, or any suitable device. The computer-executable componentcan be a processor but any suitable dedicated hardware device can(alternatively or additionally) execute the instructions.

As a person skilled in the art will recognize from the previous detaileddescription and from the figures and claims, modifications and changescan be made to the embodiments of the invention without departing fromthe scope of this invention as defined in the following claims.

I claim:
 1. A method for transferring control of an autonomous vehicleto a remote operator comprising: accessing a corpus of driving recordsof a fleet of autonomous vehicles operating within a geographic region;identifying a road segment, within the geographic region, associatedwith a frequency of transitions, from autonomous operation to localmanual operation triggered by local operators occupying autonomousvehicles in the fleet, that exceeds a threshold frequency based on thecorpus of driving records; associating a location of the road segment,represented in a navigation map, with a remote operator trigger; and atthe autonomous vehicle operating within the geographic region:autonomously navigating along a route; in response to approaching thelocation associated with the remote operator trigger: transmitting arequest for manual assistance to the remote operator; transmittingsensor data to a remote operator portal associated with the remoteoperator; and executing a navigational command received from the remoteoperator via the remote operator portal; and resuming autonomousnavigation along the route after passing the location.
 2. The method ofclaim 1: wherein accessing the corpus of driving records of the fleet ofautonomous vehicles comprises accessing geospatial locations, within thegeographic region, of instances of transition from autonomous operationto local manual operation triggered by local operators occupyingautonomous vehicles in the fleet; and wherein identifying the roadsegment comprises: aggregating instances of transition from autonomousoperation to manual operation into a set of groups based on geospatialproximity, the set of groups comprising a first group containinginstances of transition at geospatial locations along the road segment;for the first group, calculating the frequency of transitions based on aratio of quantity of transitions in the first group to quantity ofinstances of autonomous vehicles in the fleet traversing the roadsegment; and flagging the road segment for the remote operator triggerin response to the frequency of transitions exceeding the thresholdfrequency.
 3. The method of claim 2: wherein accessing the corpus ofdriving records of the fleet of autonomous vehicles comprises accessingtimes of instances of transition from autonomous operation to manualoperation; wherein aggregating instances of transition from autonomousoperation to manual operation into the set of groups comprisesaggregating instances of transition from autonomous operation to manualoperation into the set of groups further based on temporal proximity,the first group containing instances of transition at geospatiallocations along the road segment within a daily time window; furthercomprising limiting the remote operator trigger for the road segmentaccording to the daily time window; and wherein transmitting the requestfor manual assistance to the remote operator comprises, at theautonomous vehicle, transmitting the request for manual assistance tothe remote operator in response to approaching the location of theremote operator trigger during the daily time window.
 4. The method ofclaim 1: wherein accessing the corpus of driving records of the fleet ofautonomous vehicles comprises accessing geospatial locations, within thegeographic region, of instances of transition from autonomous operationto local manual operation triggered by local operators occupyingautonomous vehicles in the fleet and scene characteristics proximalautonomous vehicles during instances of transition from autonomousoperation to manual operation; wherein identifying the road segmentcomprises: aggregating instances of transition from autonomous operationto manual operation into a set of groups based on geospatial proximityand similarity of scene characteristics, the set of groups comprising afirst group containing instances of transition at geospatial locationsalong the road segment and associated with a particular scenecharacteristic; and flagging the road segment for the remote operatortrigger in response to the frequency of transitions in the first groupexceeding the threshold frequency; further comprising limiting theremote operator trigger for the road segment according to the particularscene characteristic; and wherein transmitting the request for manualassistance to the remote operator comprises, at the autonomous vehicle,transmitting the request for manual assistance to the remote operator inresponse to detecting the particular scene characteristic whileapproaching the location of the remote operator trigger.
 5. The methodof claim 4: wherein accessing scene characteristics proximal autonomousvehicles during instances of transition from autonomous operation tomanual operation comprises accessing scene characteristics comprisingoffsets between anteroposterior axes of autonomous vehicles and the Sunduring instances of transition from autonomous operation to manualoperation; wherein identifying the road segment comprises identifyingthe first group containing instances of transition at geospatiallocations along the road segment concurrent with offsets, betweenanteroposterior axes of autonomous vehicles and the Sun, within a solaroffset window; wherein limiting the remote operator trigger for the roadsegment according to the particular scene characteristic compriseslimiting the remote operator trigger for the road segment according tothe solar offset window; and wherein transmitting the request for manualassistance to the remote operator comprises, at the autonomous vehicle,transmitting the request for manual assistance to the remote operator inresponse to an offset between an anteroposterior axes of the autonomousvehicle and the Sun falling within the solar offset window whileapproaching the location of the remote operator trigger.
 6. The methodof claim 4: wherein accessing scene characteristics proximal autonomousvehicles during instances of transition from autonomous operation tomanual operation comprises accessing scene characteristics comprisingpresence of pedestrians proximal autonomous vehicles during instances oftransition from autonomous operation to manual operation; whereinidentifying the road segment comprises identifying the first groupcontaining instances of transition at geospatial locations along theroad segment concurrent with presence of a minimum quantity ofpedestrians; wherein limiting the remote operator trigger for the roadsegment according to the particular scene characteristic comprisesassociating the remote operator trigger for the road segment withpresence of the minimum quantity of pedestrians; and whereintransmitting the request for manual assistance to the remote operatorcomprises, at the autonomous vehicle, transmitting the request formanual assistance to the remote operator in response to detecting morethan the minimum quantity of pedestrians proximal the autonomous vehiclewhile approaching the location of the remote operator trigger.
 7. Themethod of claim 1: wherein autonomously navigating along the routecomprises, at the autonomous vehicle, autonomously navigating from apickup location to a drop-off location specified by a user while theuser occupies the autonomous vehicle and while a local operator isabsent from the autonomous vehicle; wherein transmitting the request formanual assistance to the remote operator comprises transmitting therequest for manual assistance to the remote operator in response toapproaching the location associated with the remote operator triggerwhile a local operator is absent from the autonomous vehicle; andfurther comprising, at a second autonomous vehicle occupied by a secondlocal operator: autonomously navigating along a second route; inresponse to approaching the location associated with the remote operatortrigger while occupied by the second local operator, prompting thesecond local operator to assume manual control of the second autonomousvehicle; and in response to passing the location, prompting the localoperator to confirm autonomous navigation of the second autonomousvehicle along the second route.
 8. The method of claim 1: whereintransmitting the request for manual assistance to the remote operatorcomprises, transmitting the request for manual assistance to a remoteoperator manager in response to approaching the location associated withthe remote operator trigger; at the remote operator manager: in responseto receiving the request for manual assistance from the autonomousvehicle, selecting the remote operator from a set of available remoteoperators; and routing sensor data received from the autonomous vehicleto the remote operator portal, associated with the remote operator, viaa computer network.
 9. The method of claim 1: further comprising, at theautonomous vehicle, estimating a time of arrival of the autonomousvehicle at the location associated with the remote operator triggerwhile autonomously navigating along the route; wherein transmitting therequest for manual assistance to the remote operator comprises, at theautonomous vehicle, transmitting the request for manual assistance tothe remote operator in response to the time of arrival falling below athreshold duration at a first time; wherein transmitting sensor data tothe remote operator portal comprises, at the autonomous vehicle,streaming sensor data to the remote operator portal in real-time over awireless computer network following the first time; and furthercomprising ceasing transmission of sensor data to the remote operatorportal after passing the location.
 10. The method of claim 1: furthercomprising autonomously slowing to a stop in response to approaching thelocation associated with the remote operator trigger; whereintransmitting the request for manual assistance to the remote operatorcomprises transmitting the request for manual confirmation to resumeautonomous navigation along the route; and wherein executing thenavigational command received from the remote operator comprisesresuming autonomous navigation along the route past the location inresponse to receipt of manual confirmation from the remote operator. 11.method of claim 1: further comprising, in response to confirmation ofmanual assistance from the remote operator, transferring braking,acceleration, and steering controls of the autonomous vehicle to theremote operator portal; wherein executing the navigational commandreceived from the remote operator comprises executing braking,acceleration, and steering commands received from the remote operatorportal; and wherein resuming autonomous navigation along the routecomprises resuming autonomous navigation along the route in response toreceipt of confirmation from the remote operator to resume autonomousnavigation.
 12. The method of claim 1, further comprising: accessing acorpus of historical traffic accident data of manually-operated vehiclesinvolved in traffic accidents within the geographic region; identifyinga second road segment, within the geographic region, associated with afrequency of traffic accidents that exceeds a second threshold frequencybased on the corpus of historical traffic accident data; associating asecond location of the second road segment, in the navigation map, witha second remote operator trigger; and at the autonomous vehicle:autonomously navigating along a second route; in response to approachingthe second location associated with the second remote operator trigger:transmitting a second request for manual assistance to a second remoteoperator; transmitting sensor data to a second remote operator portalassociated with the second remote operator; and executing a secondnavigational command received from the second remote operator via thesecond remote operator portal; and resuming autonomous navigation alongthe second route after passing the second location.
 13. The method ofclaim 1: wherein autonomously navigating along the route comprises, atthe autonomous vehicle, autonomously navigating along the route based onan autonomous navigation model; further comprising, at a remote computersystem: recording a corpus of sensor data received from the autonomousvehicle following the request for manual assistance; recording thenavigational command entered by the remote operator and served to theautonomous vehicle responsive to the request for manual assistance;generating a revised autonomous navigation model based on the corpus ofsensor data and the navigational command; and loading the revisedautonomous navigation model onto the autonomous vehicle.
 14. The methodof claim 1, further comprising: deriving a set of characteristics of theroad segment; scanning the navigation map for a second road segmentexhibiting characteristics similar to the set of characteristics of theroad segment; associating a second location of the second road segment,in the navigation map, with a second remote operator trigger; and at theautonomous vehicle: autonomously navigating along a second route; inresponse to approaching the second location associated with the secondremote operator trigger: transmitting a second request for manualassistance to a second remote operator; transmitting sensor data to asecond remote operator portal associated with the second remoteoperator; and executing a second navigational command received from thesecond remote operator via the second remote operator portal; andresuming autonomous navigation along the second route after passing thesecond location.
 15. The method of claim 1, further comprising:identifying a second road segment, within the geographic region,associated with a second frequency of transitions, from autonomousoperation to remote manual operation triggered by autonomous vehicles inthe fleet, that exceeds a second threshold frequency based on the corpusof driving records; associating a second location of the second roadsegment, represented in the navigation map, with a second remoteoperator trigger; and at the autonomous vehicle: autonomously navigatingalong a second route; in response to approaching the second locationassociated with the second remote operator trigger: transmitting asecond request for manual assistance to a second remote operator;transmitting sensor data to a second remote operator portal associatedwith the second remote operator; and executing a second navigationalcommand received from the second remote operator via the second remoteoperator portal; and resuming autonomous navigation along the secondroute after passing the second location.
 16. A method for transferringcontrol of an autonomous vehicle to a remote operator comprising:accessing a corpus of historical traffic accident data ofmanually-operated vehicles involved in traffic accidents within ageographic region; identifying a road segment, within the geographicregion, associated with a frequency of traffic accidents that exceeds athreshold frequency based on the corpus of historical traffic accidentdata; associating a location of the road segment, represented in anavigation map, with a remote operator trigger; and at the autonomousvehicle operating within the geographic region: autonomously navigatingalong a route; in response to approaching the location associated withthe remote operator trigger: transmitting a request for manualassistance to the remote operator; transmitting sensor data to a remoteoperator portal associated with the remote operator; and executing anavigational command received from the remote operator via the remoteoperator portal; and resuming autonomous navigation along the routeafter passing the location.
 17. A method for transferring control of anautonomous vehicle to a remote operator comprising: accessing aspecification for triggering manual control of autonomous vehicles;identifying a road segment, within a geographic region, exhibitingcharacteristics defined by the specification; associating a location ofthe road segment, represented in a navigation map, with a remoteoperator trigger; and at the autonomous vehicle operating within thegeographic region: autonomously navigating along a route; in response toapproaching the location associated with the remote operator trigger:transmitting a request for manual assistance to the remote operator;transmitting sensor data to a remote operator portal associated with theremote operator; and executing a navigational command received from theremote operator via the remote operator portal; and resuming autonomousnavigation along the route after passing the location.
 18. The method ofclaim 17: wherein accessing the specification for triggering manualcontrol of autonomous vehicles comprises accessing a threshold frequencyof traffic accidents per unit distance; wherein identifying the roadsegment comprises: accessing a corpus of historical traffic accidentdata of manually-operated vehicles involved in traffic accidents withinthe geographic region; based on the corpus of historical trafficaccident data, isolating the road segment associated with a frequency ofhistorical traffic accidents exceeding the threshold frequency.
 19. Themethod of claim 17: wherein accessing the specification for triggeringmanual control of autonomous vehicles comprises accessing a thresholdfrequency of transitions, from autonomous operation to remote manualoperation, triggered by autonomous vehicles; wherein identifying theroad segment comprises: accessing a corpus of driving records of a fleetof autonomous vehicles operating within the geographic region; isolatingthe road segment, within the geographic region, associated with afrequency of transitions, from autonomous operation to local manualoperation triggered by autonomous vehicles in the fleet, that exceedsthe threshold frequency of transitions based on the corpus of drivingrecords.
 20. The method of claim 17: wherein accessing the specificationfor triggering manual control of autonomous vehicles comprises accessinga threshold frequency of transitions, from autonomous operation toremote manual operation, triggered by local operators occupyingautonomous vehicles; wherein identifying the road segment comprises:accessing a corpus of driving records of a fleet of autonomous vehiclesoperating within the geographic region; isolating the road segment,within the geographic region, associated with a frequency oftransitions, from autonomous operation to local manual operationtriggered by local operators occupying autonomous vehicles in the fleet,that exceeds the threshold frequency of transitions based on the corpusof driving records.